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ABSTRACT 

Previously, we reported qualitative findings showing that students who expe- 
rienced a problem- or enquiry-based course (EBL) in a first-year seminar pro- 
gram had greater confidence in their academic abilities, were more engaged, 
and were better prepared for upper-year courses. In the current paper, we 
provide quantitative data to substantiate the students’ qualitative conclusions. 
We present results to show that these students do perform at a significantly 
higher level compared with members of the control group who did not experi- 
ence an EBL course. Using survey data, we show that the EBL students shift 
the way they access information compared with peers: they preferentially use 
more sophisticated resources for research. At the same time, students report 
greater engagement in the community, and student engagement is known to 
contribute to increased academic performance. 

RESUME 

Dans un article precedent, nous avons rapporte les resultats de nos analyses 
qualitatives, qui demontraient que les etudiants inscrits dans un programme de 
premiere annee utilisant une methode d’apprentissage par explication ou par 
Taction (enquiry-based learning ou EBL) avaient une plus grande confiance 
en leurs capacites scolaires, etaient plus engages socialement et se sentaient 
mieux prepares pour les cours de niveau superieur. Dans l’article qui suit, nous 
presentons des donnees quantitatives pour justifier les conclusions subjectives 
des etudiants. Nos donnees demontrent une performance nettement plus 
elevee de ces etudiants que ceux du groupe temoin, qui n’employaient pas la 
methode d’apprentissage par Taction. Grace aux donnees recueillies a meme 
le sondage, nous demontrons que les etudiants du groupe dont l’apprentissage 


CJHE / RCES Volume 40, No. 2, 2010 


Engagement, Performance and Enquiry-based Learning / A. Summerlee & J. Murray 79 


etait base sur l’action modifient la fa^on dont ils acquierent l’information, 
contrairement aux etudiants du groupe controle. En effet, les etudiants du 
premier groupe preferent recourir a des ressources plus sophistiquees pour 
leurs recherches. Ils rapportent aussi une plus grande participation dans la 
collectivite, une activite qui contribue a ameliorer la performance academique. 
Enfm, les auteurs suggerent que l’expression « apprentissage par Taction » est 
plus appropriee pour decrire cette methode d’apprentissage . 


INTRODUCTION 

There are strong imperatives to change the undergraduate experience in universi- 
ties. The advent of the Internet, which has made information ubiquitously available, 
ought to change our approaches to teaching and learning. There is a body of evidence 
that demonstrates that students’ active engagement in their postsecondary institu- 
tion is one of the key drivers for academic success (Kuh, 2003). At the same time, the 
learning styles and attitudes of so-called millennial students are different from those 
of previous generations. To remain effective, universities need to accelerate their pace 
of evolution to accommodate to these changes. 

In 2007, we reported that introducing problem-based learning in a first-year semi- 
nar course at the University of Guelph significantly affected the learning behaviours 
of students, excited in them greater motivation to succeed, and led to enhanced rea- 
soning and processing skills that were both transferable and persistent throughout 
the students’ undergraduate experience (Murray & Summerlee, 2007). The changes we 
documented in that paper were self-reported observations by students and based on 
qualitative observation. 

The purpose of the present paper is to provide quantitative evidence for the persis- 
tent effects of problem-based learning, when offered as an elective in the first year of 
university. We followed the progress of students who participated in the problem-based 
seminar courses through the remainder of their academic program to determine if their 
own beliefs about the increase in their performance could be substantiated by measur- 
able changes in their academic performance as revealed by grades in other courses. In 
addition, we undertook a systematic study of the students’ approach to the learning 
resources that they used during their seminars to determine whether changes in their 
academic performance were related to, or could be attributed to, differences in their 
skills at scholarly research, in particular by accessing more appropriate resources. 

Research also points to a reciprocal relationship between engagement in the class- 
room and effective learning (Kuh, 2003), and between active learning and engagement 
in the university or wider community (Kuh, 2005a). There are also data to suggest that 
students in small classes show a greater level of community engagement (Ahlfeldt, 
Mehta, & Sellnow, 2005). Based on these findings, we also explored whether students 
who became engaged in learning through a problem-based course might also show a 
greater level of community engagement. 

Definition of the Problem-Based Experience 

We developed a suite of seminars that use a specific approach to teaching and 
learning known as closed-loop reiterative problem-based learning (Barrows, 1986; 
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Schmidt, 1983; Murray & Summerlee, 2007: see definition below). Although there has 
been analysis of the effectiveness of problem-based learning (Albanese, 1993; Vernon 
& Blake, 1993 ; Ahlfeldt et ah, 2005), comparisons are confounded by the use of differ- 
ent approaches to problem-based learning. In some cases, the term is loosely applied to 
diverse classroom approaches where problems are used to illustrate principles or ideas: 
in other situations it is used to refer to situations where problems are used to stimulate 
learning. Despite a number of specific publications focused on this approach, there 
remains a remarkably low level of acceptance of the fact that closed-loop reiterative 
problem-based learning is quite a different pedagogy from the simple use of problems 
in large classes to demonstrate or reinforce the principles conveyed in a traditional 
lecture format. Part of the challenge may come from the confusion created by the use 
of the word “problem” in the phrase “problem-based learning,” because it seems to 
emphasize the centrality of the “problem.” It implies that there is a problem for which 
students must find the answer when the situation is quite the contrary. Closed-loop 
reiterative problem-based learning uses a “problem” or a “situation” to incite students 
to question context, to find information that supports understanding the principles 
that lie behind the problem, and to reflect upon the wider implications. The problem, 
better termed scenario, is used to pique students’ interest and motivate them to enquire 
about the underlying issues. 

The authors, therefore, strongly recommend that the discourse adopt the language 
of enquiry-based learning (EBL), a term that better reflects the emphasis of the peda- 
gogy. This will shift the pedagogical focus to enquiry and away from problem solving. 
Even this change of language may not provide sufficient clarity. It is, therefore, impor- 
tant that we are precise about the nature of the educational experience in the research 
that we are reporting in this paper. 

In closed-loop reiterative EBL, students are presented with a scenario in their first 
session. They are required to identify issues, research the principles underlying the 
issues, and learn material in context. (For full details of the approach, see Murray & 
Summerlee, 2007). It is not necessary for the students to solve the problems; in fact, 
many of the scenarios are deliberately designed so they do not have a ready solution. 
The scenarios are intended to motivate the students to engage in learning and under- 
stand the issues that underlie the scenario. Students research their learning issues and 
return to the second class to share information and integrate new knowledge into the 
scenario; this process may be repeated. The reiterative nature of the process encourag- 
es students to practise effective communication and to learn how to criticize and how 
to behave in academic situations, in addition to learning and assimilating content. The 
students are encouraged to find appropriate resources that lead to understanding, and 
they share information and resources with each other. 

At the end of every class meeting, time is devoted to group processing. This is 
an absolutely crucial part of the learning process and is often avoided by other peda- 
gogical approaches that have adopted the label “problem-based learning.” In group 
processing, each member of the group provides concise and precise feedback to every 
other member of the group, as well as engaging in self-assessment of their own perfor- 
mance. This provides the opportunity for the students and the facilitators to enhance 
communication, critical thinking, and research skills. One third of eveiy class period 
is spent in group processing. 
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Study Groups 

We have explored the application and longer-term implications of EBL in a num- 
ber of different areas over the past five years. This has involved students taking one 
of three different first-year seminar courses taught in an EBL mode: Sex, Gender and 
Sexuality; Politics, Science and the Environment; and Confronting Cultural Dilemmas. 
All three courses are interdisciplinary and we have completed more than ten seminar 
groups comprising close to 100 students in total. The students crossed the full spectrum 
of undergraduate programs and disciplines at the university. Upon completion of the 
EBL course, students continued their studies in traditional courses and programs, and 
the data in the present paper are based on their performance, during the remainder of 
their studies, compared with that of students in control groups. 

METHODOLOGY 

There were four objectives to the research: (1) to follow the academic progress 
of students who had completed an EBL first-year seminar (determined by analysis of 
their grades) compared with students in the control groups; (2) to explore whether the 
observed differences in grade performance could be associated with the average grades 
of students upon entering university; (3) to investigate the impact of the EBL seminar 
on the types of resources that students used in their research; and (4) to study whether 
or not there was a correlation between the level of volunteerism and community en- 
gagement of the EBL students. 

Objective 1 : To Follow Academic Performance of Students 
Who Completed an EBL Seminar Course 

To follow the academic progress of students who completed an EBL seminar course 
on their subsequent performance in more traditional courses, the grades of a group 
of seventeen students who had taken the EBL course called Sex, Gender and Sexual- 
ity were tracked throughout their undergraduate program. Every semester following 
the EBL seminar, the grade averages of this group of students was compared with two 
control groups: one, a group of students who had participated in a different first-year 
seminar (i.e., they had elected to have a small group experience but were not taught in 
an EBL mode), and the second, a group of students chosen from a course delivered in 
traditional large-lecture format. The students in the two control groups were matched 
by gender and grade average upon entrance to university with students in the experi- 
mental group. Statistical differences in performance were determined using ANOVA 
and compared by students’ t test scores. 

This review of performance was repeated with a second group of students who had 
taken an EBL seminar called Politics, Science, and the Environment. There were nine 
students in this group. The goals were to determine whether an EBL course would have 
repeatable impact on subsequent performance with a different set of students and to 
determine whether the students’ performance was affected by the subject matter of the 
course, or, more likely, was attributable to the pedagogical approach. 
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Objective 2: To Determine Whether Entering Grades 
Were Correlated to Subsequent Performance 

To determine whether the entering grades of students correlated with the overall 
impact of the EBL course on subsequent performance, students who participated in 
enquiry-based learning courses were divided into four groups according to their aca- 
demic record upon entrance to university (admission average) : 

• Stratum 1 - students with entering grades of 71-75 per cent (n = 16) 

• Stratum 2 - students with entering grades of 76-80 per cent (n = 29) 

• Stratum 3 - students with entering grades of 81-85 per cent (n = 55) 

• Stratum 4 - students with entering grades of >85 per cent (n = 24). 

The performance of these four strata was compared at the end of their sixth semes- 
ter of study (3 years of undergraduate courses) in two ways: (1) the average grade in 
all courses completed, and (2) the average grade improvement by the end of semester 
six, compared with the entering average. 

Objective 3: To Investigate Whether the EBL Course 
Changed the Students’ Approaches to Resources/Research Materials 

To study the immediate impact of the enquiry-based seminar, students (n = 18) 
completed a survey on their use and access to resources before and immediately after 
completion of the EBL course in their first year. The questions in the survey focused on 
three areas: (1) access to electronic resources; (2) access to scholarly journals, encyclo- 
pedias and specialized websites; and (3) access to human resources (faculty, librarians, 
staff, and fellow students). The questions are shown in Table 1. Students ranked the 
frequency with which they accessed different types of resources on a five-point scale 
from never to always, and the results from before and after the course were compared 
to highlight any shifts in patterns of behaviour. 

A second survey was designed to identify persistence in the use and access to 
resources. The survey was administered again to the same group of students (n = 18), 
upon completion of their second year of study (4 semesters). The instrument was also 
completed by a control group of students (n = 18) who had taken a non-EBL first-year 
seminar. The results of the surveys were collated and compared. Statistical differences 
in the reported use of resources were determined using ANOVA and compared by stu- 
dents’ t test scores. 

Objective 4: To Study Whether Participation in an EBL Course 
Influenced Engagement 

There is a wealth of literature on engagement (Ahlfeldt et al., 2005; Kuh, 2003; 
Kuh & Hayek, 2004). The measure of engagement in the learning process in many cir- 
cumstances is judged by asking students how many times they participate in class dis- 
cussion, work in small groups, and challenge each other’s assertions and observations. 
In an EBL class, students would expect to score highly on these types of surveys so we 
sought an additional measure of engagement. We argue that time spent volunteering 
and participating in an international experience during the summer could serve as a 
measure of student engagement. The responses of the same two groups of students (n 
= 18) surveyed for Objective 3 were compared using x 2 analysis. Students were asked 
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to complete two surveys: one at the start of semester two (before the first-year semi- 
nar) and one after completion of the second year (four semesters) of study. The survey 
asked students to identify how much time per week they spent volunteering inside and 
outside the institution, and to report if they had participated in an international expe- 
rience during the summer between their first and second years of study. 

RESULTS 

Objective 1: To Determine the Impact of an EBL Course 
on Subsequent Performance 

With their permission, a cohort of 13 students was followed from semester two 
until graduation. Of those 13 students, two were in a science program, two in com- 
merce, two in arts, two in applied science, two in environmental science, and three in 
the combined arts and science program. 



Semester level (standard 4 year - 8 semester program) 

Figure 1. Average grade in all courses completed. 

Notes: The impact of one first-year course using enquiry-based learning (EBL - black diamonds, n = 13) 
on the average grades during the remainder of the course of study compared with a control group of stu- 
dents in a self-selected seminar taught in a more traditional mode (Non-EBL - black squares, n = 13) and 
a control group of students (grey triangles - n = 13) taught in standard didactic lectures. The mean and 
standard error of the mean is shown for each group. There is a trend for the students in the EBL group to 
have higher average grades after their experience in the EBL class. 


The performance of the 13 students in the EBL group was compared with the two 
control groups (students who were matched for gender, program of study, and enter- 
ing grade); the results are shown in Figure 1. It is notable that the students in both 
first-year Sex, Gender and Sexuality seminars performed better during their second 
semester, i.e., when they were working in small groups, compared with students in 
traditional classes, but, more significantly, the EBL group showed statistically superior 
performance on average in subsequent semesters compared with members of the con- 
trol groups. By graduation, the difference between these groups had increased to an 
average of 8.9 per cent grade points (range 2.5 - 11.2 per cent). 

A second group of nine students, who were in the Politics, Science and Environ- 
ment seminar, were followed to determine whether the subject matter of the course 
influenced the outcomes. Of this group, three students were in science, two in envi- 
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ronmental science, one in business, one in arts, and two in the combined arts and 
science program. In semester six, the students demonstrated statistically improved 
performance compared with the control groups, similar to the finding for the first 
cohort studied. 


Objective 2: To Determine Whether Entering Grades 
Affect Subsequent Performance 


The academic performance of all students who have taken an EBL course (n = 106 
students) was evaluated at the end of their third year of university (after completing 
semester 6). The students were divided into four groups, according to their grades upon 
admission to university. Their overall academic average for all courses completed by 
the end of semester 6 is shown in Figure 2. 



Figure 2. Average grade at end of year 3 

Note: The impact of one first-year course using enquiry-based learning (EBL) on the average grades during 
courses taken in the second year of undergraduate study compared with a control group of students in a 
small-group seminar course not taught in an enquiry-based learning mode. Students were stratified into 
four levels according to their grade average on admission to university. Performance is shown in two ways: 
the average grade performance for each stratum after completing three years of study shown in black dia- 
monds and the actual grade improvement (comparison of end year three grades and entering grades) shown 
in grey squares. Note the greater improvement by students with the lowest entry grades but all groups show 
an overall improvement. 


The average grades for students in all four strata improved, as would be anticipat- 
ed based on the data reported in Figure 1. To explore this effect further, we analyzed 
the change in grade performance between grades upon entry to university and grades 
at the end of the third year of study (upon completion of six semesters). The results 
appear in Figure 2. The data reveal that the improvement in performance is greatest 
for those students with the lowest grades upon entry to university. 

Objective 3: To Investigate Whether the EBF Seminar 
Changed the Students’ Approaches to Resources/Research Materials 

Eighteen students completed a survey on their use of research resources before 
they started and immediately upon completion of an EBL seminar. Students ranked 
the frequency of use of particular types of resources on a five-point scale from never 
to frequently in three areas: (1) access to electronic resources; (2) access to scholarly 
journals, encyclopedias, and specialized websites; and (3) access to human resources 
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(faculty, librarians, staff, and fellow students). There were significant shifts in the pat- 
tern of access to resources. The results are shown in Table 1. 

Table 1. 

Changing Patterns in Accessing Research Resources. 

The patterns of access to resource materials by students (n = 18) were assessed by 
surveys immediately before and after completion of an enquiry-based course. Signifi- 
cantly greater or less use of specific resources is shown. 



Greater 

Less 


Use 

Use 

Access to electronic search agents and resources 

Internet search engines (Google, Yahoo etc.) 

No change 


Wikipedia 

- 

p < 0.05 

Online data bases 

p < 0.05 

- 


Access to scholarly journals, encyclopedias and specialized websites 


Original scholarly articles 
Specialized websites of research institutions 
General encyclopedias 
Special encyclopedias 

p < 0.01 

p < 0.05 
p < 0.05 
p < 0.05 

Access to personnel as resources 

Reference librarians 

p < 0.01 

Professors 

p < 0.05 

Teaching assistants 

p < 0.05 

Family and friends 

p < 0.01 

Fellow students 

No change 


The EBL students shifted away from a relatively naive mode of research that relied 
upon Wikipedia and talking to professors, teachers, and family as important resources. 
They developed a more sophisticated research repertoire including consulting scholarly 
articles, specialized websites of research institutions, general and specific encyclope- 
dias, and online data bases. The specific shift to consulting scholarly articles is striking. 
A comparison of sources consulted before and after the course appears in Figure 3. 

Students appear to have developed more independent research capability and con- 
fidence because they relied less on family, friends, professors, and teachers. Moreover, 
they reported a significant increase in the frequency with which they consulted refer- 
ence librarians to help to identify appropriate sources. 

Extending this study further, we explored the persistence of different patterns of 
accessing resources as students progressed through their programs of study. We com- 
pared the patterns of access to information between the same group of students (EBL 
course) surveyed above, with a control group (students who had not completed an 
EBL course in their first year). The control group was matched for entrance grades and 
gender with the experimental group. The results of the comparison of their responses 
are shown in Table 2. The EBL group used Wikipedia, professors, and teachers less fre- 
quently as a resource compared with the control group. They continued preferentially 
to use scholarly articles, more sophisticated data-bases and encyclopedias, and refer- 
ence librarians as resources. 
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Figure 3. The impact of an enquiry-based course on access to original scholarly 
articles. 


Note: The responses of students (n = 23) before the course indicating how often they accessed journal 
articles in preparing to write an essay or make a presentation is shown in black. The response at the end of 
the course is shown in grey. There was a definite and clear shift in the behaviour of students. 


Table 2. 

Change in Patterns Comparison. 

This table compares the change in patterns of access to resource materials between 
students who experienced an enquiry-based course (EBL) with a control group of stu- 
dents who did not take an enquiry-based course (Non-EBL) (n = 18 in each group). 
Significantly greater or less use of specific resources is shown for the EBL group com- 
pared with the controls. 



Greater 

Less 

Comparison of access to resources 

use by EBL group use by EBL group 

Access to electronic search agents and resources 

Internet search agents (Google, Yahoo etc.) 


No difference 

Wikipedia 

- 

p < 0.01 

Online data bases 

p < 0.05 

- 

Access to scholarly journals, encyclopedias and specialized websites 

Original scholarly articles 

p < 0.001 

- 

Specialized websites of research institutions 

p < 0.05 

- 

General encyclopedias 


No difference 

Special encyclopedias 

p < 0.05 

- 

Access to personnel as resources 

Reference librarians 

p < 0.001 

- 

Professors 

- 

p < 0.01 

Teaching assistants 

- 

p < 0.05 

Family and friends 


No difference 

Fellow students 


No difference 
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Objective 4: To Study Whether Participation in an EBL Course 
Influenced Student Engagement 

Students reported on their level of engagement through volunteerism by complet- 
ing a survey before starting their small group experience in the second semester and 
again at the end of their second year of study. The results appear in Table 3. When 
subjected to x 2 analysis, students who participated in the EBL seminar course reported a 
significant increase in the time they spent volunteering in the community (outside the 
university), and a significant increase in the time they spent volunteering inside the 
university (in clubs, societies, and in governance processes) compared with members 
of the control group. In addition, a higher proportion of EBL students were engaged in 
international experiences during the summer vacation between their first and second 
years of study. These results are summarized in Table 3. 

Table 3. 

Student Engagement Measure. 

A measure of engagement of students. Engagement of students in an enquiry-based seminar (EBL) is 
compared with those who did not participate (non-EBL). 



EBL Group (n = 56) 

Non-EBL group (n = 56) 

Start semester 1 

End semester 4 

Start semester 1 

End semester 4 

Total time volunteering (h/wk) 

3.4 ± 0.7 

5.7 ± 0.3* 

3.3 ± 0.6 

4.6 + 0.6 

Within the university (h) 

1.6 ± 0.3 

2.8 ± 0.1* 

1.7 ± 0.2 

2.0 ± 0.3 

Outside the university (h) 

1.8 ± 0.2 

2.9 ± 0.3* 

1.6 ± 0.4 

2.6 ± 0.3 

International experience summer 

- 

92»/o 

- 

66°/o 


Note: * indicates significant difference. 


DISCUSSION 

We set out to examine quantitative outcomes from using EBL as part of the first- 
year experience at the University of Guelph. We have already reported on qualitative 
outcomes (self-reflection by students) of the impact of such a course and disseminated 
those results. In that research, students reported that they not only learned content, 
they also learned processing and reasoning skills that enhanced their learning experi- 
ences in other courses and gave them the confidence to believe that they were more 
effective and responsible learners (Murray & Summerlee, 2007). In the present paper, 
we report that those students showed a superior level of performance in subsequent 
courses, had different patterns for accessing research resources compared to the con- 
trol group, and had a higher level of engagement in their university and community. 
Moreover, the effects of EBL on academic performance appear to be inversely cor- 
related with academic achievement prior to entering university: students with lower 
entering grades showed the greatest percentage increase in grades-based performance 
compared with their peers in the control groups and compared to their EBL peers with 
higher entering grades. 


Impact on Performance 

Students in two different EBL classes showed superior performance after first year 
when compared to control groups (Figure 1). At one level, it might be argued that 
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the higher rate of academic achievement in the small group seminars is explained by 
the fact these seminars serve as learning communities. Alexander Meiklejohn (Smith, 
2001) first introduced this concept in his short-lived experimental college in Wisconsin 
in the 1920s. He argued that students who engaged in learning together - effectively 
a learning community - showed superior academic performance when compared to 
students learning alone. The concept re-emerged in the late 1980s, supported by the 
growing recognition that student engagement in educationally purposeful activities, 
inside and outside the classroom, is a precursor to higher levels of achievement (Kuh, 
1996; 2003; Zhao & Kuh, 2004). Lenning and Ebbers (1999) recognized four generic 
learning communities: curricular communities (students co-enrolled in two or more 
courses linked by a common theme), classroom learning communities (group process 
learning in a classroom setting), residential learning communities (students in resi- 
dence clusters taking two or more classes together), and student-type learning com- 
munities (targeted groups of similar academic achievement and/or underrepresented 
groups). Students in the EBL seminars certainly fit within the concept of classroom 
as learning community. However, while the shared academic experience in the class 
may be part of the basis for the immediate success experienced by the EBL students, 
learning together in a small group is not an adequate explanation for their persistently 
enhanced success. In subsequent semesters, students from the EBL group consistently 
performed better than their peers in the control groups, including a control group 
of students who selected different small group seminars but were not taught in an 
enquiry-based mode. Moreover, the EBL group showed higher performance for the re- 
mainder of their undergraduate experience, compared to the didactically taught group 
(Figure 1), something students in the non-EBL small group did not do. 

Theoretically, learning communities are intentionally structured to help students 
make two types of links. The first is to encourage them to connect ideas from different 
disciplines; the second is to forge enduring social connections within the group that 
will allow them to develop their own identity and voice within the group. These tenets 
are fundamental underpinnings of closed-loop reiterative EBL and are practised and 
reinforced throughout the EBL courses. This focus alone, however, does not seem suffi- 
cient explanation for the persistence of the superior academic performance by the EBL 
groups. We have already reported that students in EBL seminars show a superior level 
of motivation and have greater confidence in their ability and responsibility to carry 
out independent learning (Murray & Summerlee, 2007). Data presented in the present 
paper demonstrate that, working in an enquiry-based mode, students refine their ap- 
proach to research (Table 1): they seek more sophisticated resources, rely less on fam- 
ily and friends for information, rely less on Wikipedia, and more frequently consult 
scholarly papers and articles. There was also a notable increase in the use of reference 
librarians as resources to help students find materials. This supports the contention of 
Kuh and Gonyea (2003) about the increasingly important role that libraries and librar- 
ians must play in student engagement and support for learning. Moreover, these shifts 
in patterns of information seeking differentiate the EBL students from their colleagues 
(Table 2). A comparison of the ways students sought information in their third year, 
highlights significant differences between EBL students and their peers. It is possible 
that the higher academic performance in the EBL groups is related to a combination 
of factors: being part of a tight-knit learning community, being more motivated to 
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find out and learn, and having developed the research skills and the understanding of 
the importance of research, to be able to find information to support their learning. 
Finally, students reported increased confidence in their own abilities (Murray & Sum- 
merlee, 2007), recognized the limits of their knowledge, and had the ability to find 
information efficiently compared to non-EBL students. These may also be important 
factors in driving more effective learning. 

There has been a considerable amount of research, especially in the United States, 
about the effect of academic selectivity on the undergraduate student body at so- 
called quality institutions (Bowen & Bok, 1998; Kuh & Pascarella, 2004; Pascarella & 
Terenzini, 1991; Pascarella et al., 2006; Rumberger & Thomas, 1993; Thomas, 2003). 
Although there are variations in the definition of selectivity in admissions, the gener- 
ally accepted surrogate for quality is the notion that the average or median score of 
entering or enrolled students, on standardized tests such as the ACT or SAT, reflects 
academic ability and drives a higher level of quality in a given institution (Bowen & 
Bok, 1998; Flowers, Osterlind, Pascarella, & Pearson, 2001; Pascarella et al., 2006). The 
logic underlying this belief, at first glance, appears reasonable: the interaction among 
students of comparable abilities constitutes a dimension of institutional impact (Kuh, 
Shuh, & Whitt, 1991; however, it is difficult to find supporting evidence. In fact, such 
beliefs belie the significance of the diversity of backgrounds, experiences, and abilities 
that underpin much of society’s critical drive for greater inclusiveness (Bridges, Cam- 
bridge, Kuh, & Leegwater, 2005; Flarper, Carini, Bridges, & Flayek, 2004; Kuh & Um- 
back, 2005). In other words, entering grades are used as a convenient proxy for qual- 
ity, and an institution with a higher grade-average cut-off is deemed to be of higher 
quality, although there is little evidence to support this conclusion (Ehrenberg, 2003). 
There have been three major studies on the impact of admission grades on a variety of 
alumni outcomes (Bowen, 1977; Pascarella & Terenzini, 1991; Pascarella et al., 2006) 
that have come to two fundamentally similar conclusions. First, there is a positive 
link between entering grades and career success (in particular, earnings). So if quality 
is measured in economic terms, there may be an argument to support the contention 
that the higher the entry grades, the better students will perform and, concomitantly, 
the higher the quality of the institution. The second finding is that there is very little 
evidence that students’ cognitive, developmental, or psychosocial outcomes are cor- 
related with entering grades: the correlation at most institutions is very weak and there 
are huge variations between educational establishments. In other words, these studies 
imply that educational experience can influence student performance and is by no 
means necessarily related to the starting point (that is, grades on entry). 

Data in the current paper support this latter conclusion. The overall level of su- 
perior performance was not positively correlated with entering grades; in fact, it ap- 
pears to be the reverse. The number of students studied here is not sufficient to allow 
for statistically significant correlations, however, the trend is very clear: all students 
in the EBL seminar groups performed better, on average, in their subsequent courses, 
compared to their peers in the control groups, and the effect on students with lower 
entering grades was greatest. Perhaps this finding is not surprising; academic high 
flyers do not have as much leeway for grade improvement. Nevertheless, the self-re- 
flections suggest that all students experience a greater level of satisfaction with their 
educational experience and increased confidence as they move forward (Murray & 
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Summerlee, 2007). Moreover, students come to appreciate that all their colleagues, of 
whatever academic, social, ethnic, or economic background, enhance their learning 
experience. 

In a meta-analysis of engagement by students, where varying levels of problem- 
based learning (PBL) methodology were used, Ahlfeldt and colleagues (2005) demon- 
strated that engagement increases in three circumstances: (1) as course level increases; 
(2) as class size decreases; and (3) as the component of PBL increases. Their analysis 
was based on responses to questions in the National Survey of Student Engagement 
(NSSE) by students from a number of institutions practising various forms of PBL. As 
we mentioned above, there are numerous different forms of pedagogical approach 
encompassed in the term PBL. Nevertheless, Ahlfeldt and colleagues (2005) were able 
to demonstrate statistically significant improvement in engagement among students 
involved in small-group active learning. The inference is that greater engagement will 
lead to higher academic performance and greater satisfaction with the educational 
experience. 

The data presented by Ahlfeldt and colleagues also suggest that PBL classes are 
more effective at upper levels of undergraduate study. The implication is that upper 
year students are more likely to have the skills to be able to take advantage of PBL. The 
data presented in the present paper, however, suggests otherwise. In the circumstances 
that we examined, students with an EBL experience in their first year performed bet- 
ter than their peers (Figure 1), were more engaged in the community (Table 3), and 
demonstrated research skills that set them apart from their peers (Tables 1 and 2). It 
is tempting to suggest that the rigorous approach to EBL that we implement in the 
classroom, and report on in this paper, accounts for these differences. It is possible that 
there is a correlation between participation in the EBL seminar and subsequent perfor- 
mance, but the data sets are not adequate to substantiate this observation. 

Both Ahlfeldt et al. (2005) and Murray and Summerlee (2007) have analyzed the 
relative success of PBL based on self-reporting by students (completing surveys). The 
quantitative data provided in this paper augments and confirms the conclusions of 
student self-reflection. Additionally, and perhaps more persuasively for faculty who 
are concerned about the apparent lack of sequence and structure in enquiry- or prob- 
lem-based learning, the data demonstrate that students also develop conventional in- 
tellectual rigor. 


Impact on Engagement 

The literature around engagement and academic performance is compelling (Ahl- 
feldt et al., 2005; Kuh, 2003 ;Kuh & Hayek, 2004), and in the present study, the students 
in the EBL seminars report a higher level of engagement. The literature on engagement 
has tended to rely on information gathered by the National Survey of Student Engage- 
ment (NSSE), which asks students to respond to questions such as how often they have 
asked questions in class, worked with other students, worked with classmates outside 
class, or tutored other students? (Ahlfeldt et al., 2005). These sorts of questions address 
the fundamentals of EBL, so it would be reasonable to expect that the meta-analysis 
of performance of EBL/PBL students would reveal greater levels of engagement. At the 
same time, the students are asked questions about analyzing, synthesizing, and orga- 
nizing data and applying information. Again, the latter are absolutely critical elements 
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of PBL/EBL (Barrows, 1986; Murray & Summerlee, 2007) so we would anticipate that 
under these criteria for student engagement, students in PBL/EBL classes would score 
highly. Consequently, to avoid this potential predictability, we chose to use a different 
definition of engagement: participation in volunteer activity in the community and 
participation in an international experience in the summer break between first and 
second year of university. Students at the University of Guelph already have a very 
high level of volunteer activity compared with university students across Canada: the 
participation rate is 70 per cent while the Canadian average for volunteer activity is 40 
per cent. Nevertheless, as Table 5 indicates, students in the EBL seminar groups showed 
an increase in overall level of volunteer activity after completing the seminar class, 
and a greater proportion chose an international summer experience. In the survey, 
students had the opportunity to provide commentary on their level of participation. 
Some of the comments reflect a clear understanding of the value of engagement to 
overall self-awareness: “I learned so much from volunteering that I decided to expand 
my horizons and visit another country,” and “I realized that I was discovering so much 
about myself as I volunteered - I was motivated to do more and I found I learned so 
much more.” One potential drawback in the way we sought to establish participation 
could be that volunteering and participating in an international experience may not 
be available to all students: students who lacked financial means may not have had 
time to volunteer owing to work commitments, or they may have been unable to af- 
ford an international experience. None of the students reported this as an impediment 
to participation in either volunteer activities or an international experience, and the 
same concern would have applied to both the EBL and the non-EBL group. It is also 
possible that students who chose the EBL seminars were already more confident and 
this might affect the engagement results but this could not be tested. 


In conclusion, the observations reported in this paper support the qualitative data 
previously published: students show superior academic performance after completing 
a first-year experience that truly engages them in learning and in learning about how 
to learn. Such an experience builds their confidence, but more importantly, it changes 
their approach to research and to the use of resources. The experience of EBL provides 
a context for students to develop the skills necessary to cope more effectively with 
subsequent material presented in traditional formats. These results raise intriguing 
questions about how to redesign the first-year experience for university students to 
create a more engaged cohort of learners.* 
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